System for constructing virtual closet and creating coordinated combination, and method therefor

ABSTRACT

A system for constructing a virtual closet according to an embodiment of the present disclosure includes: a user terminal detecting an image including a user&#39;s face, extracting a region corresponding to at least one fashion item in the detected image, restoring the fashion item from the region corresponding to the fashion item, collecting a fashion image matched to the restored fashion item, and extracting and transmitting a characteristic of the collected fashion image and a creation date of the detected image to a server; and the server analyzing a wearing date and wearing frequency of the fashion item on the basis of the extracted date, detecting all images of the user wearing the fashion item, and storing the extracted characteristic of the fashion image, the wearing date of the fashion item, and the images of the user wearing the fashion item together with the fashion image.

FIELD

The present disclosure relates to a system and method for constructing avirtual closet and creating a coordinated combination, and in moredetail, a system and method for constructing a virtual closet andcreating a coordinated combination, the system and method constructing avirtual closet on the basis of fashion items of a user and creating acoordinated combination by performing machine learning on the basis ofinput images.

BACKGROUND

The demands for personalized services are explosively increasing withthe development of big data and artificial intelligence technologies. Apersonalized service, which provides a service that a user would preferby analyzing the individual taste and desire of the user, can be appliedto almost all fields including food, movie, traveling, and news. Afashion industry is also a field to which personalized services havebeen considerably applied, and the personalized services are used in theway of recommending products, which are expected to be preferred andvery likely to be purchased by users, of recommending relevant shoppingmalls, or the like.

It is necessary to sufficiently find out the characteristic of each userin order to provide a good personalized service, but it is verydifficult to find out individual fashion information in the fashionfield. For example, it is substantially impossible for fashion serviceproviders to obtain information about what kinds of clothes individualshave, what clothes individuals frequently wear or not, the body sizesand a fit preference of users, etc. Accordingly, most fashionpersonalized services at most recommend fashion products by estimatingthe tastes of users on the basis of indirect information of the userssuch as age, sex, search history, etc.

Information about clothes of users, what coordination the users makewith the clothes, what clothes they wore, when they wore the clothes,and what styles they created is core data that are the base of anexcellent fashion personalized service. Accordingly, there were manyattempts to obtain such user closet data. For example, “smart closet”,“your closet”, etc. that are smartphone applications that were releasedin 2016 and are being serviced at present provide a function enablingindividuals to make their own virtual closets.

However, users can construct their own virtual closets only when theytake pictures of hundreds of clothes in person, remove the backgroundsin the pictures in person, and input various items of information aboutthe clothes. Further, users have to record one by one when they wore theclothes, how they coordinate, etc. in the calendar of the application inperson.

Accordingly, these services failed to collect many users due to thetrouble that users have to take pictures of clothes one by one inperson, remove the backgrounds in the pictures, and input various itemsof information in person. Further, many similar services that enableusers to manage closets in the field of online exist, but most users areindifferent to those services because it is very difficult andtroublesome to construct a virtual closet.

SUMMARY

The present disclosure has been made in an effort to solve the problemsdescribed above and an objective of the present disclosure is to providea system and method for constructing a virtual closet and creating acoordinated combination, the system and method automatically extractingand storing fashion items of a user and creating and storing acoordinated set.

Another objective of the present disclosure is to provide a system andmethod for constructing a virtual closet and creating a coordinatedcombination, the system and method being able to create and recommend acoordinated combination in consideration of the fashion items of a user.

In order to achieve the objectives, a system for constructing a virtualcloset according to an embodiment of the present disclosure includes: auser terminal detecting an image including a user's face, extracting aregion corresponding to at least one fashion item in the detected image,restoring the fashion item from the region corresponding to the fashionitem, collecting a fashion image matched to the restored fashion item,and extracting and transmitting a characteristic of the collectedfashion image and a creation date of the detected image to a server; andthe server analyzing a wearing date and wearing frequency of the fashionitem on the basis of the extracted date, detecting all images of theuser wearing the fashion item, and storing the extracted characteristicof the fashion image, the wearing date of the fashion item, and theimages of the user wearing the fashion item together with the fashionimage.

According to various embodiments of the present disclosure, sincefashion items of a user are automatically stored, it is possible toprovide a service that a user can more conveniently use and propose acoordinated set on the basis of the fashion items of the user.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a configuration diagram of a system for constructing a virtualcloset and creating a coordinated combination according to an embodimentof the present disclosure.

FIG. 2 is a diagram showing detailed data processing of the system forconstructing a virtual closet and creating a coordinated combinationaccording to an embodiment of the present disclosure.

FIG. 3 is a diagram illustrating a process of extracting fashion itemsaccording to an embodiment of the present disclosure.

FIG. 4 is a diagram illustrating a process of constructing a virtualcloset according to an embodiment of the present disclosure.

FIG. 5 is a diagram illustrating virtual closet construction accordingto another embodiment of the present disclosure.

FIG. 6 is a flowchart illustrating a method of controlling a system forconstructing a virtual closet and creating a coordinated combinationaccording to an embodiment of the present disclosure.

FIG. 7 is a diagram showing the detailed configuration of a serveraccording to an embodiment of the present disclosure.

FIG. 8 is a diagram about software modules stored in a storage unitaccording to an embodiment of the present disclosure.

FIG. 9 is a diagram about software modules stored in a storage unitaccording to another embodiment of the present disclosure.

FIG. 10 is a configuration diagram of a system for constructing avirtual closet and creating a coordinated combination according toanother embodiment of the present disclosure.

FIG. 11 is a diagram showing detailed data processing in the system andmethod for constructing a virtual closet and creating a coordinatedcombination according to an embodiment of the present disclosure.

FIG. 12 is a diagram illustrating a process of creating user informationaccording to an embodiment of the present disclosure.

FIG. 13 is a diagram illustrating a process of acquiring fashioninformation according to an embodiment of the present disclosure.

FIG. 14 is a diagram illustrating a process of creating a coordinatedcombination according to a first shopping type according to anembodiment of the present disclosure.

FIG. 15 is a diagram illustrating a process of creating a coordinatedcombination according to a second shopping type according to anembodiment of the present disclosure.

FIG. 16 is a diagram illustrating a method of creating a coordinatedcombination according to an embodiment of the present disclosure.

FIG. 17 is a diagram showing the detailed configuration of a serveraccording to another embodiment of the present disclosure.

FIG. 18 is a diagram about software modules stored in a storage unitaccording to an embodiment of the present disclosure.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

The present disclosure is described hereafter in detail with referenceto the accompanying drawings. Further, in describing the presentdisclosure, detailed descriptions of well-known functions orconfigurations relating to the present disclosure will not be providedso as not to obscure the description of the present disclosure withunnecessary details. Further, the following terminologies are defined inconsideration of the functions in the present disclosure and may beconstrued in different ways by the intention or practice of users andoperators. Therefore, the definitions thereof should be construed basedon the contents throughout the specification.

FIG. 1 is a configuration diagram of a system for constructing a virtualcloset and creating a coordinated combination according to an embodimentof the present disclosure.

Referring to FIG. 1 , a system 10 for constructing a virtual closet andcreating a coordinated combination according to an embodiment of thepresent disclosure includes a user terminal 100 and a sever 200.

In the present disclosure, the system 10 for constructing a virtualcloset and creating a coordinated combination means a system thatconstructs a virtual closet by automatically storing fashion items of auser, and creates, stores, and provides a coordinated set to a user onthe basis of the fashion items of the user, and the fashion item mayinclude not only clothing, fashion sundry products, and accessories, butalso electronic devices or wearable devices that can be worn on a bodyas fashion items.

Further, the user terminal 100 may be implemented as a smartphone, amobile device, etc., but is not limited thereto and may be implementedas various electronic devices such as a PDA, a pad type PC, a tablet PC,a wearable device, etc.

The user terminal 100 detects an image including a user's face, extractsa region corresponding to at least one fashion item in the detectedimage, restores the fashion item from the region corresponding to thefashion item, collects a fashion image matched to the restored fashionitem, and extracts and transmits the characteristic of the collectedfashion image and the creation date of the detected image to the server200.

In particular, the user terminal 100 can receive images that are inputthrough various routes, and in detail, the user terminal 100 may includea route of obtaining images by photographing through a lens, a route ofdownloading pictures through a social media, a route of downloadingpictures stored in a storage medium such as a USB, etc.

The user terminal 100 can recognize faces in images input throughvarious routes and classify the images in accordance with the faces. Theuser terminal 100 proposes frequently appearing faces to a user and whenthe user selects his/her face, the user terminal 100 detects an imageincluding the selected user's face.

The user terminal 100 extracts a region corresponding to at least onefashion item that the user wears in the detected image.

The user terminal 100 automatically removes the background except forthe fashion item from the extracted region corresponding to theextracted fashion item. The user terminal 100 determines whether an itemhaving high similarity has been registered in a virtual closet on thebasis of the fashion item with the background removed.

If an item that is highly similar to the fashion item with thebackground removed has been registered in a virtual closet, there is noneed for a specific fashion item registration process. However, when anitem that is highly similar to the fashion item with the backgroundremoved has not been registered in a virtual closet, the user terminal100 performs a fashion item registration process.

In detail, the user terminal 100 restores a fashion item into a completeimage. In general, this is because when a region corresponding to afashion item that a user wears in a picture, a partial region of thefashion item is hidden by other fashion items or objects in many cases.

In this case, the user terminal 100 can create and obtain a completeimage from a partial image of the fashion item through a trainedGenerative Adversarial Networks (GAN) model.

Alternatively, the user terminal 100 can search for and provide afashion item having the highest similarity by comparing a partial imageof the fashion item with fashion items stored in a database.

Meanwhile, the user terminal 100 collects a fashion image matched to therestored fashion item, and extracts and transmits the characteristic ofthe collected fashion image and the creation date of the detected imageto the server 200.

In detail, the user terminal 100 can extract the characteristic of afashion image matched to the restored fashion item, that is, thecharacteristic of a fashion image such as the kind, color, material,sense of season, ability to keep warm, etc. of the fashion item.

The user terminal 100 can extract the creation date of the detectedimage by extracting the information of the date on which the detectedimage including the user's face was taken.

The user terminal 100 can transmit the extracted characteristic of thefashion image and the creation date of the detected image to the server200.

Meanwhile, the server 200 according to an embodiment of the presentdisclosure analyzes the wearing date and the wearing frequency of thefashion item on the basis of the extracted date, detects all images ofthe user wearing the fashion item, and stores the extractedcharacteristic of the fashion image, the wearing date of the fashionitem, and images of the user wearing the fashion item together with thefashion image.

In detail, the server 200 stores the fashion image matched to thecompleted restored fashion item and the characteristic of the fashionimage together in a virtual closet.

The server 200 can analyze the date on which images including the user'sface were taken, automatically record that the user wearing fashionitems in the images wore the fashion items on the date, and detectwearing frequency by analyzing how many times the user wore the fashionitems for a specific period.

The server 200 detects all images taken with the user wearing each ofthe fashion items.

The server 200 stores the characteristics of the extracted fashionimages, the wearing dates of the fashion items, and images of the userwearing the fashion items together with the fashion images in a virtualcloset.

Accordingly, the server 200 can automatically store fashion imagescorresponding to the fashion items of the user in a virtual closettogether with the characteristics of the fashion images, the wearingdates of the fashion items, and all images of the user wearing thefashion items.

Meanwhile, the server 200 according to an embodiment of the presentdisclosure can create a coordinated set image by grouping at least onefashion item included in a detected image, and can store the wearingdate and frequency of the grouped fashion item, an image of the userwearing the grouped fashion item, and the created coordinated settogether in a virtual closet.

In detail, the server 200 can analyze at least one fashion item that auser simultaneously wears in one detected image, automatically configurea coordinated set with the fashion item, and store an image of thecoordinated set configured in this way in a virtual closet.

The server 200 can analyze the date on which an image including a user'sface was taken, automatically record that the user wearing thecoordinated set in the image wore the coordinated set on the date, anddetect wearing frequency by analyzing how many times the user wore thecoordinated set for a specific period.

The server 200 detects all images taken with the user wearing eachcoordinated set.

The server 200 stores the wearing date and wearing frequency of acoordinated set, an image of the user wearing the coordinated set, and acoordinated set image together in a virtual closet.

Accordingly, the server 200 can automatically store an image of acoordinated set configured on the basis of the fashion items of the userin the virtual closet together the wearing date and wearing frequency ofthe coordinated set, and all images of the user wearing the coordinatedset.

The server 200 provides the information stored with the fashion item,that is, the characteristic of the fashion image, the wearing date ofthe fashion item, and all images of the user wearing the fashion item,and the information stored with the coordinated set, that is, thewearing date and wearing frequency of the coordinated set and all imagesof the user wearing the coordinated set to the user terminal 100.

FIG. 2 is a diagram showing detailed data processing of the system forconstructing a virtual closet and creating a coordinated combinationaccording to an embodiment of the present disclosure.

Referring to FIG. 2 , the user terminal 100, as described above, canreceive various pictures input from a terminal, a social media account,and other storages, recognize a user in the input various pictures, andsort images including the user.

The user terminal 100 can extract a region of each of all fashion itemsincluded in the pictures, and automatically remove a background in eachof the extracted regions.

The user terminal 100 analyzes similarity between images registered in avirtual closet of the user and the extracted fashion items. When afashion item is not searched for from the registered images due to lowsimilarity, the user terminal 100 can restore the region of a fashionitem into a complete image and extract the properties of the fashionitem from the region of the restored fashion item.

The user terminal 100 can extract and transmit the extracted propertiesof the fashion item and the creation date of the image related to thefashion item to the server 200.

Meanwhile, the server 200 can store and register property data of aclothing image, the picture from which clothing was extracted, andinformation of using the clothing on the date on the basis of thereceived properties of the fashion items and the creation date of theimage related to the fashion item.

The server 200 can store and register a coordinated set composed of atleast one fashion item included in one detected image including theuser's face, the picture from which the coordinated set was extracted,and the information of using the coordinated set at a date.

Meanwhile, the server 200 can create and provide a trendy fashion itemor coordinated set to the user on the basis of fashion items andcoordinated sets registered in an individual virtual closet of the user,product information acquired from an external shopping mall server,fashion information acquired from a social network, etc. through anartificial intelligence module.

The server 200 can recommend a product that is similar in style,feeling, color, etc. to a fashion item that a user has in a largequantity or frequently uses, or a product that can be coordinated withthe fashion item on the basis of information registered in a virtualcloset through the artificial intelligence module.

FIG. 3 is a diagram illustrating a process of extracting fashion itemsaccording to an embodiment of the present disclosure.

Referring to FIG. 3 , the user terminal 100 extracts regions 310, 320,and 330 corresponding to fashion items from an input image 300 andautomatically removes the backgrounds except for the fashion items fromthe extracted regions, thereby being able to extract clean images 311,321, and 331 of only the fashion items.

The user terminal 100 analyzes the characteristic of each of theextracted images 311, 321, and 331. For example, the user terminal 100analyzes an extracted image 311 that the kind is an upper garment, thematerial is cotton, and the style is a short sleeve shirt, and transmitsthe analyzed characteristics to the server 200 together with theextracted images 311, 321, and 331, and the server 200 stores thereceived extracted images 311, 321, and 331 and the receivedcharacteristic of the extracted image in a virtual closet.

FIG. 4 is a diagram illustrating a process of constructing a virtualcloset according to an embodiment of the present disclosure.

Referring to FIG. 4 , a plurality of images input through various routesis stored in the user terminal 100 (410), and the user terminal 100detects an image including the user's face (420), extracts regionscorresponding to at least one fashion item (sunglasses, an uppergarment, a lower garment, shoes, and a bag) in the detected image (430),and restores the fashion items into complete images by automaticallyremoving the backgrounds in the extracted regions corresponding to thefashion items (sunglasses, an upper garment, a lower garment, shoes, anda bag) (S440).

The user terminal 100 collects fashion images matched to the restoredfashion items (450) and detects characteristics of the collected fashionimages (460). As the characteristics of collected fashion images, akind, a color, a pattern, a sleeve, a neckline, a material, a sense oflength, etc. are shown in FIG. 4 .

When the user terminal 100 extracts and transmits the characteristics ofthe collected fashion images and the creation date of the detected imageto the server 200, the server 200 automatically registers the extractedcharacteristics of the fashion images, the wearing date of the fashionitems, etc. in a virtual closet together with the fashion images (470).

The server 200 creates and registers a coordinated set image configuredby grouping at least one fashion item included in the detected userimage (480).

When receiving regions corresponding to fashion items and the creationdate of a detected image from the user terminal 100, the server 200 cananalyze and update them in real time. That is, when receiving a regioncorresponding to a new fashion item and the creation date of a detectedimage from the user terminal 100, the server 200 analyzes and storesthem in a virtual closet, thereby being able to update the virtualcloset.

The user terminal 100 can extract purchase information and images ofpurchase products from a purchase history image and transmit them to theserver 200.

FIG. 5 is a diagram illustrating virtual closet construction accordingto another embodiment of the present disclosure.

Referring to FIG. 5 , the user terminal 100 extracts product images 520,530, and 540 from a purchase history image 510 and transmits informationsuch as the selected size, color information, cost, purchase date,quantity, etc. to the server 200.

The server 200 can extract the characteristics of the images of thepurchase products and can store the extracted characteristics of thepurchase products and the purchase information together with the imagesof the purchase products. In detail, the server 200 accesses a shoppingmall that sells a purchase product and searches for the purchaseproduct, and extracts various items of information such as arepresentative image and the specification of the searched purchaseproduct.

The server 200 automatically removes the background of the extractedrepresentative image, analyzes similarity to fashion items registered ina virtual closet, stops a registration process when a fashion itemhaving high similarity has been registered in the virtual closet, andperforms a registration process if the fashion item having highsimilarity is not registered.

Meanwhile, the server 200 may collect a fashion image corresponding to afashion item, detect a characteristic, extract a date from a detectedimage including a user's face, and store them in a virtual closet, asdescribed above, may extract a purchase product image from a purchasehistory image and store the purchase product image together withpurchase information in a virtual closet, and may store them in avirtual closet by performing processes the same as those described aboveon a detected image including a user's face and a purchase history imagefor supplementation.

In detail, the server 200 analyzes similarity between a fashion item anda purchase product, replaces the fashion item with the purchase product,and stores information stored with the fashion item, purchaseinformation, and the purchase product together when the similarity is apreset reference or higher.

This is because an image acquired in relation to a purchase product andthe specification of the purchase product are generally likely to bemore accurate.

The server 200 may extract a kind, a color, a material, etc. from afashion image corresponding to a fashion item in a detected imageincluding a user's face and store them in a virtual closet, and mayextract information such as a size, a price, a purchase date, etc. froma purchase history image and store them in a virtual closet.

When a kind, a color, a material, etc. extracted from a purchase historyimage are clearer from those extracted from a fashion imagecorresponding to a fashion item, the server 200 may substitute and storethe kind, color, material, etc. extracted from the purchase historyimage.

Meanwhile, the server 200 can collect fashion information on the basisof preference and the number of times of sharing on a social network,create a coordinated combination composed of at least one of fashionitems and purchase products stored through the system for constructing avirtual closet on the basis of the collected fashion information, andprovide the coordinated combination to the user terminal 100.

That is, the server 200 can collect fashion information on the basis ofpreference and the number of times of sharing of various fashion imagesacquired from a social network, a shopping mall server, etc. and candetermine fashion information over a preset level in accordance with thecollected fashion information through an artificial intelligence module.

The server 200 can provide a trendy coordinated combination, which iscreated by combining and configuring at least one of fashion items andpurchase products stored through the system for constructing a virtualcloset on the basis of the collected fashion information, to the userterminal 100.

FIG. 6 is a flowchart illustrating a method of controlling a system forconstructing a virtual closet and creating a coordinated combinationaccording to an embodiment of the present disclosure.

Referring to FIG. 6 , a method of controlling a system for constructinga virtual closet and creating a coordinated combination according to anembodiment of the present disclosure includes: a step in which the userterminal 100 detects an image including a user's face and detects aregion corresponding to at least one fashion item in the detected image(S610); a step in which the user terminal 100 restores the fashion itemfrom the region corresponding to the fashion item and collects a fashionimage matched to the restored fashion item (S620); a step in which theuser terminal 100 extracts and transmits the characteristic of thecollected fashion image and the creation date of the detected image tothe server 200 (S630); a step in which the server 200 analyzes thewearing date and wearing frequency of the fashion item on the basis ofthe extracted date (S640); a step in which the server 200 detects allimages of the user wearing the fashion item (S650); and a step in whichthe server 200 stores the extracted characteristic of the fashion image,the wearing date of the fashion item, and the images of the user wearingthe fashion item together with the fashion image (S660).

The method of controlling a system for constructing a virtual closet andcreating a coordinated combination according to an embodiment of thepresent disclosure may further include a step of creating a coordinatedset image by grouping at least one fashion item included in the detectedimage, and storing the wearing date and frequency of the grouped fashionitem, an images of the user wearing the grouped fashion item, and thecreated coordinated set in a virtual closet.

The method of controlling a system for constructing a virtual closet andcreating a coordinated combination according to an embodiment of thepresent disclosure may further include a step of providing informationstored with the fashion item and information stored with the coordinatedset image to the user terminal.

The method of controlling a system for constructing a virtual closet andcreating a coordinated combination according to an embodiment of thepresent disclosure may further include a step of analyzing and updatingthe region corresponding to the fashion item and the creation date ofthe detected image when the region and the creation date are receivedfrom the user terminal.

The method of controlling a system for constructing a virtual closet andcreating a coordinated combination according to an embodiment of thepresent disclosure may further include a step of extracting purchaseinformation and an image of a purchase product from a purchase historyimage and transmitting the purchase information and the image of thepurchase product to the server.

The method of controlling a system for constructing a virtual closet andcreating a coordinated combination according to an embodiment of thepresent disclosure may further include a step in which the serverextracts the characteristic of the image of the purchase product andstores the extracted characteristic of the purchase product and thepurchase information together.

The method of controlling a system for constructing a virtual closet andcreating a coordinated combination according to an embodiment of thepresent disclosure may further include a step in which the serveranalyzes similarity between the fashion item and the purchase product,replaces the fashion item with the purchase product when the similarityis over a preset level, and stores the information stored with thefashion item, the purchase information, and the purchase producttogether.

The method of controlling a system for constructing a virtual closet andcreating a coordinated combination according to an embodiment of thepresent disclosure may further include a step in which the servercollects fashion information on the basis of preference and the numberof times of sharing on a social network, and creates a coordinatedcombination composed of at least one of fashion items and purchaseproducts stored through the system for constructing a virtual closet onthe basis of the collected fashion information, and provides thecoordinated combination to the user terminal.

FIG. 7 is a diagram showing the entire configuration of a serveraccording to an embodiment of the present disclosure.

Although FIG. 7 shows the entire configuration of the server 700, theuser terminal 100 may also be shown in a similar configuration, and theconfiguration and functions described in relation to FIG. 7 may equallybe applied to the user terminal 100.

The server 200 includes a communication unit 210, a processor 220, and astorage 230.

The processor 220 generally controls operation of the server 200 foranalyzing the wearing date and the wearing frequency of a fashion itemon the basis of an extracted date, detecting all images of a userwearing the fashion item, and storing an extracted characteristic of afashion image, the wearing date of the fashion item, and images of theuser wearing the fashion item together with the fashion image.

In detail, the processor 220 includes a RAM 221, a ROM 222, a main CPU223, a graphic processing unit 224, first to n-th interfaces225-1˜225-n, and a bus 226.

The RAM 221, the ROM 222, the main CPU 223, the graphic processing unit224, the first to n-th interfaces 225-1˜225-n, etc. may be connected toeach other through the bus 226.

The first to n-th interfaces 225-1˜225-n are connected to the variouscomponents described above. One of the interfaces may be a networkinterface connected to an external device through a network.

The main CPU 223 accesses the storage 230 and performs booting using anO/S stored in the storage 230. Further, the main CPU 223 performsvarious operations using various programs, contents, data, etc. storedin the storage 230.

In particular, the main CPU 223 may analyze the wearing date and thewearing frequency of a fashion item on the basis of an extracted date,detect all images of a user wearing the fashion item, and store anextracted characteristic of a fashion image, the wearing date of thefashion item, and images of the user wearing the fashion item togetherwith the fashion image in a virtual closet.

A set of commands for booting the system, etc. are stored in the ROM222. When a turning-on instruction is input and power is supplied, themain CPU 223 copies the O/S stored in the storage to the RAM 221 andoperates the O/S in accordance with the commands stored in the ROM 222,thereby booting the system. When booting is finished, the main CPU 223copies various application programs stored in the storage 230 to the RAM221 and executes the application programs copied to the RAM 221, therebyperforming various operations.

The graphing processing unit 224 creates an image including variousobjects such as an icon, an image, a text, etc. using an operation unit(not shown) and a rendering unit (not shown). The rendering unit (notshown) computes property values such as coordinate values, shapes,sizes, colors, etc. for displaying the objects in accordance with thelayout of the image on the basis of a received control instruction. Animage of various layouts including the objects is created on the basisof the property values computed by rendering unit (not shown) and theoperation unit (not shown).

In particular, the graphic processing unit 224 can implement an objectcreated by the main CPU 223 into a Graphic User Interface (GPU), anicon, a user interface image, etc.

Meanwhile, the operations of the processor 220 described above may beperformed by programs stored in the storage 230.

The storage 230 stores an Operating System (O/S) module and various datasuch as various multimedia contents for analyzing the wearing date andthe wearing frequency of a fashion item on the basis of an extracteddate, detecting all images of a user wearing the fashion item, andstoring an extracted characteristic of a fashion image, the wearing dateof the fashion item, and images of the user wearing the fashion itemtogether with the fashion image.

In particular, the storage 230 may include a software module foranalyzing the wearing date and the wearing frequency of a fashion itemon the basis of an extracted date, detecting all images of a userwearing the fashion item, and storing an extracted characteristic of afashion image, the wearing date of the fashion item, and images of theuser wearing the fashion item together with the fashion image.

FIG. 8 is a diagram about software modules stored in a storage unitaccording to an embodiment of the present disclosure.

Referring to FIG. 8 , programs such as a date/frequency analysis module231, an image detection module 232, and a storage module 223 may bestored in the storage 230 of the server 200.

Meanwhile, the operations of the processor 220 described above may beperformed by programs stored in the storage 230. Hereafter, detailedoperations of the processor 220 using programs stored in the storage 230are described in detail.

In detail, the date/frequency analysis module 231, on the basis ofinformation of the date on which an image including a user's face wastaken, can consider a fashion item or a coordinated set included in theimage as being worn on the date and analyze the date on which thefashion item or the coordinated set was worn, or the wearing frequencyfor a specific period.

The image detection module 232 can detect all images of a user wearing aspecific item.

The storage module 233 can store the characteristic of an extractedfashion image, the wearing date of a fashion item, and images of a userwearing the fashion item together with the fashion image.

FIG. 9 is a diagram about software modules stored in a storage unitaccording to another embodiment of the present disclosure.

Referring to FIG. 9 , the storage 230 included in the user terminal 100is exemplified, and a region extraction module 231 can extract a regioncorresponding to at least one fashion item from a detected imageincluding a user's face.

A collection module 232 can restore the fashion item from the regioncorresponding to the fashion item and collect a fashion image matched tothe restored fashion item.

A transmission module 233 can extract and transmit the characteristic ofthe collected fashion image and the creation date of the detected imageto the server.

FIG. 10 is a configuration diagram of a system for constructing avirtual closet and creating a coordinated combination according toanother embodiment of the present disclosure.

In this embodiment, a system 100 for constructing a virtual closet andcreating a coordinated combination may mean a system for automaticallycreating and providing various coordinated combinations that can berecommended to a user, the target of coordination may include not onlyclothing, fashion sundry products, and accessories, but also electronicdevices or wearable devices that can be worn on a body as fashion items.

A user terminal 100 can extract a region corresponding to a fashion itemfrom an input image, and can create and transmit user informationincluding at least one of the characteristic of the extracted region, auser shopping type, and user coordination conditions to a server 200.

In this case, there are various routes through which images are inputfor a coordination combination. In detail, the route may include a routeor uploading a picture obtained by photographing through a lens, a routeof uploading a picture on a social media, a route of uploading a picturestored in a storage medium such as a USB, etc.

As described above, when various images for a coordinated combination,the user terminal 100 can extract regions corresponding to a fashionitem shown in the images.

For example, when a piece of trousers is shown in an image, it ispossible to extract a region in which the piece of trousers is shown.

The user terminal 100 can extract the characteristic of an extractedregion, in detail, can acquire only an image related to a fashion itemby automatically removing the other background except for the fashionitem and extract the characteristic of the extracted region by analyzingthe acquired image of the fashion item.

In detail, the characteristic of an extracted region includesinformation about at least one of the kind, color, material, and styleof a fashion item.

For example, the kind of a fashion item means whether the fashion itemis an upper garment or a lower garment, the color mans a color, thematerial means whether it is cotton, and the style means whether it is asportswear or a suit.

The user terminal 100 creates and transmits user information includingat least one of the characteristic of the extracted region, and a usershopping type and user coordination conditions input from the user tothe server 200.

The coordination conditions mean a guideline preset in relation tocoordination by the user, may be set as a wearing purpose, a wearingsituation, an atmosphere, weather, temperature, etc., and for example,when hot summer and exercise are set as coordination conditions, acoordinated combination is created for a sportswear that can be worn inhot summer.

Meanwhile, the user terminal 100 can receive a shopping type from auser, and the user shopping type includes a first shopping typeaccording to a coordinated combination created by combination with atleast one of product information and fashion information on the basis ofan input image, and a second shopping type according to a coordinatedcombination created by combining images input on the basis ofinformation selected by a user from product information and purchaseinformation. This will be described below.

The server 200 extracts images corresponding to a fashion item fromproduct information collected from a shopping mall server and fashioninformation collected on a social network, and analyzes the propertiesthereof.

In detail, the server 200 can collect property information such as aproduct image and the specification of the product uploaded on analigned shopping mall from the shopping mall, and can collect an imageof the product itself by removing the background in the product imageuploaded on the shopping mall.

The server 200 can collect fashion information on the basis ofpreference and the number of times of sharing on a social network.

The server 200 extracts an image corresponding to the fashion item fromthe collected product information and fashion information and analyzesthe properties thereof. For example, the server 200 analyzes theproperties of a product on the basis of collected product information,that is, the specification of a product uploaded to a shopping mall,extracts an image, which corresponds to clothing or a fashion sundryproduct that an influencer or a celebrity wears on a social network,from the clothing or fashion sundry product, and analyzes the propertiesof thereof.

The server can determine a fashion combination over a preset level byperforming machine learning on the extracted image and the analyzedproperties. The server 200 can create a coordinated combination matchedto a fashion combination determined on the basis of at least one ofproduct information and fashion information, and user information.

In detail, in relation to an extracted image and analyzed properties,the server 200 can detect the kind of a fashion item included in animage, analyze and store properties such as the color, material, style,the degree of keeping warm, etc. of the detected fashion item, canacquire information about what the newest trend is in accordance withreferences such as what fashion items are the most preferred, whatfashion items are the most shared on social networks, what fashion itemsare the most exposed to advertisements, etc. on the basis of storeddata.

The server 200 can determine a fashion combination over a presetreference, that is, a fashion combination that coincides with the newesttrend in accordance with the information about the acquired newesttrend. Such fashion combination may be one fashion item or may include acombination of a plurality of fashion items.

The server 200 can create a coordinated combination configured such thatat least one of user information received from the user terminal 100,product information acquired from a shopping mall server, and fashioninformation acquired from a social network is matched to the determinedfashion combination.

In accordance with the types described above, the server 200 can createa trendy fashion style by appropriately combining images uploaded by auser, that is, images related to fashion items of a user and imagesrelated to fashion items that can be acquired from a social media or ashopping mall server.

In this case, the server 200 can determine a preset fashion combinationon the basis of preference and the number of times of sharing, and candisplay coordinated combinations in the way of creating and arrangingcoordinated combinations in order of high degree of matching with thedetermined fashion combination.

Meanwhile, the server 200 can search for a photoshoot imagecorresponding to a created coordinated combination and can providerecommendation coordination information including the searchedphotoshoot image and the created coordinated combination to the userterminal 100.

That is, the server 200 searches for a photoshoot in which a createdcoordinated combination is worn, that is, sets similarity and searchesfor photoshoots included within the similarity range, and arranges anddisplays a photoshoot, which is determined as being the most matched tothe created coordinated combination of the searched photoshoots, withthe created coordinated combination, thereby being able to providerecommendation coordination information to the user terminal 100.

The server 200 can sequentially provide the recommendation coordinationinformation from recommendation coordination information satisfying usercoordination conditions. That is, the server can include coordinationconditions input by a user, for example, a wearing purpose, a wearingsituation, an atmosphere, weather, temperature, etc., and can arrangeand display recommendation coordination information in accordance withconditions set to be prior by the user.

FIG. 11 is a diagram showing detailed data processing in the system andmethod for constructing a virtual closet and creating a coordinatedcombination according to an embodiment of the present disclosure.

Referring to FIG. 11 , the user terminal 100, as described above, canreceive a fashion item, that is, a clothing picture in FIG. 11 , from aterminal, a social media account, and other storages, automaticallyremoves a background of each input image when a user selects a shoppingtype and inputs coordination conditions on the basis of his/her clothes,extracts the properties of a fashion item in the image and stores theproperties in a user database, and simultaneously, creates and transmitsuser information including the properties of the fashion item, theselected shopping type, and the input coordination conditions to theserver 200.

The server 200 collects a trendy fashion picture through various routessuch as social media/shopping mall/fashion channel, and particularly,receives product information including an image and properties of aproduct uploaded on a shopping mall from an external server 200′ such asthe server of an aligned shopping mall.

The server 200 performs data refinement and preprocessing on the userinformation received from the user terminal 100 and the collectedfashion picture and product information, and determines a fashioncombination over a preset reference by performing fashion image analysisand learning, that is, machine learning on the data.

The server 200 creates coordination matched to the determined fashioncombination on the basis of the user information, the collected fashionpicture, and the product information, and searches for a photoshootimage corresponding to the created coordination.

The server 200 creates and provides a resultant including thecoordination created in accordance with the input coordinationconditions and the photoshoot image corresponding to the coordination tothe user terminal 100.

FIG. 12 is a diagram illustrating a process of creating user informationaccording to an embodiment of the present disclosure.

Referring to FIG. 12 , the user terminal 100 extracts regions 1210,1220, and 1230 corresponding to fashion items from an input image 1200and automatically removes the backgrounds except for the fashion itemsfrom the extracted regions, thereby being able to extract clean images1211, 1221, and 1231 of only the fashion items.

The user terminal 100 analyzes the properties of each of the extractedimages 1211, 1221, and 1231. For example, the user terminal 100 analyzesan extracted image 1211 that the kind is an upper garment, the materialis cotton, and the style is a short sleeve shirt, and stores theanalyzed properties together with the extracted images 1211, 1221, and1231 as user information.

FIG. 13 is a diagram illustrating a process of acquiring fashioninformation according to an embodiment of the present disclosure.

Referring to FIG. 13 , the server 200 collects fashion information onthe basis of preference and the number of times of sharing on a socialnetwork, and analyzes the kind, color, material, and style of a fashionitem included in the collected fashion information.

For example, the server 200 can collect fashion information bydetermining preference related to a fashion item image 1310 by searching“number of likes”, “number of emoticons for good”, “number of shares”,“related article”, etc., and can analyze the kind, color, material, andstyle of a fashion item included in the fashion item image 1310 includedin the collected fashion information.

In this process, the server 200 can determine information about whatfashion item the most coincides with the newest trend, that is, afashion combination, over a preset level by performing machine learning.

Meanwhile, as described above, a user can input a shopping type, theshopping type includes a first shopping type according to a coordinatedcombination created by combination with at least one of productinformation and fashion information on the basis of an input image inputby a user, and a second shopping type according to a coordinatedcombination created by combining images input on the basis ofinformation selected by a user from product information and purchaseinformation. In relation to this, a coordinated combination creationprocess according to each shopping type is described with reference toFIG. 5 and FIG. 6 .

FIG. 14 is a diagram illustrating a process of creating a coordinatedcombination according to a first shopping type according to anembodiment of the present disclosure.

Referring to FIG. 14 , a process of creating a coordinated combinationaccording to a first shopping type input by a user is shown.

In detail, when a user updates a trousers image 1410, the server 200 cancreate a coordinated image 1411 coinciding with a newest trend bycombining a fashion item image, which is acquired from productinformation acquired from an external server 200′ and fashioninformation acquired from a social network, with the uploaded trousersimage 1410, and can search for and provide a photoshoot image 1411-1corresponding to the created coordinated combination 1411 together withthe coordinated combination 1411.

FIG. 15 is a diagram illustrating a process of creating a coordinatedcombination according to a second shopping method according to anembodiment of the present disclosure.

Referring to FIG. 15 , a process of creating a coordinated combinationaccording to a second shopping type input by a user is shown.

In detail, when a user selects a plurality of fashion items 1520 on asocial network or a shopping mall, it is possible to create allavailable coordinated combinations 1530 by combining the selectedplurality of fashion items 1520 and the trousers image 1410 uploaded bythe user, and it is possible to arrange and display coordinatedcombinations 1531, 1532, and 1533 in high degrees of matching withfashion combinations over a preset reference determined on the basis ofpreference and the number of times of sharing.

When the user selects a coordinated combination 1531, the server 200 cansearch for and provide a photoshoot image 1531-1 corresponding to theselected coordinated combination 1531.

FIG. 16 is a diagram illustrating a method of creating a coordinatedcombination according to an embodiment of the present disclosure.

A method of creating a coordinated combination of the system forcreating a coordinated combination including a user terminal and aserver according to an embodiment of the present disclosure includes: astep in which the user terminal 100 extracts a region corresponding to afashion item from an input image, and creates and provides userinformation including at least one of the characteristic of theextracted region, a user shopping type, and user coordination conditions(S1610); a step in which the server 200 extracts an image correspondingto the fashion item from product information collected from a shoppingmall and fashion information collected from a social network, andanalyzes the properties of the image (S1620); a step in which the server200 determines a fashion combination over a preset reference byperforming machine learning on the extracted image and the analyzedproperties (S1630); and a step in which the server 200 creates acoordinated combination matched to the determined fashion combination onthe basis of at least one of the product information and the fashioninformation, and the user information (S1640).

Meanwhile, the method of creating a coordinated combination according toan embodiment of the present disclosure may further include a step inwhich the server 200 searches for a photoshoot image corresponding tothe created coordinated combination and provides recommendationcoordination information including the searched photoshoot image and thecreated coordinated combination.

In this case, the characteristic of the extracted region includesinformation about at least one of the kind, color, material, and styleof the fashion item.

The shopping type includes a first shopping type according to acoordinated combination created by combination with at least one ofproduct information and fashion information on the basis of an inputimage, and a second shopping type according to a coordinated combinationcreated by combining images input on the basis of information selectedby a user from product information and purchase information.

The step of extracting an image corresponding to the fashion item andanalyzing the properties thereof can collect fashion information on thebasis of preference and the number of times of sharing on a socialnetwork, and analyze the kind, color, material, and style of a fashionitem included in the collected fashion information.

The step of determining a fashion combination over a preset referencecan determine a fashion combination over the preset reference on thebasis of preference and the number of times of sharing, and the step ofdetermining a coordinated combination can create coordinated combinationin order or high degree of matching with the determined fashioncombination.

The step of providing recommendation coordination information to theuser terminal 100 can sequentially provide the recommendationcoordination information from recommendation coordination informationsatisfying the user coordination conditions.

FIG. 17 is a diagram showing the detailed configuration of a serveraccording to another embodiment of the present disclosure.

Although FIG. 17 shows the entire configuration of the server 700, theuser terminal 100 may also be shown in a similar configuration, and theconfiguration and functions described in relation to FIG. 17 may equallybe applied to the user terminal 100.

The server 200 includes a communication unit 210, a processor 220, and astorage 230.

The processor 220 generally controls operation of the server 200 forextracting an image corresponding to a fashion item from productinformation collected from a shopping mall server and fashioninformation collected from a social network, analyzing the propertiesthereof, determining a fashion combination over a preset reference byperforming machine learning on the extracted image and the analyzedproperties, and creating a coordinated combination matched to thedetermined fashion combination on the basis of at least one of theproduct information and the fashion information, and the userinformation.

In detail, the processor 220 includes a RAM 221, a ROM 222, a main CPU223, a graphic processor 224, first to n-th interfaces 225-1-225-n, anda bus 226.

The RAM 221, the ROM 222, the main CPU 223, the graphic processing unit224, the first to n-th interfaces 225-1-225-n, etc. may be connected toeach other through the bus 226.

The first to n-th interfaces 225-1-225-n are connected to the variouscomponents described above. One of the interfaces may be a networkinterface connected to an external device through a network.

The main CPU 223 accesses the storage 230 and performs booting using anO/S stored in the storage 230. Further, the main CPU 223 performsvarious operations using various programs, contents, data, etc. storedin the storage 230.

In particular, the main CPU 223 can extract an image corresponding to afashion item from product information collected from a shopping mallserver and fashion information collected from a social network, analyzethe properties thereof, determine a fashion combination over a presetreference by performing machine learning on the extracted image and theanalyzed properties, and create a coordinated combination matched to thedetermined fashion combination on the basis of at least one of theproduct information and the fashion information, and the userinformation.

A set of commands for booting the system, etc. are stored in the ROM222. When a turning-on instruction is input and power is supplied, themain CPU 223 copies the O/S stored in the storage to the RAM 221 andoperates the O/S in accordance with the commands stored in the ROM 222,thereby booting the system. When booting is finished, the main CPU 223copies various application programs stored in the storage 230 to the RAM221 and executes the application programs copied to the RAM 221, therebyperforming various operations.

The graphing processing unit 224 creates an image including variousobjects such as an icon, an image, a text, etc. using an operation unit(not shown) and a rendering unit (not shown). The rendering unit (notshown) computes property values such as coordinate values, shapes,sizes, colors, etc. for displaying the objects in accordance with thelayout of the image on the basis of a received control instruction. Animage of various layouts including the objects is created on the basisof the property values computed by rendering unit (not shown) and theoperation unit (not shown).

In particular, the graphic processing unit 224 can implement an objectcreated by the main CPU 223 into a Graphic User Interface (GPU), anicon, a user interface image, etc.

Meanwhile, the operation of the processor 220 described above may beperformed by programs stored in the storage 230.

The storage 230 stores various data such as an Operating system (O/S)software module, and various multimedia contents for extracting an imagecorresponding to a fashion item from product information collected froma shopping mall server and fashion information collected from a socialnetwork, analyzing the properties thereof, determining a fashioncombination over a preset reference by performing machine learning onthe extracted image and the analyzed properties, and creating acoordinated combination matched to the determined fashion combination onthe basis of at least one of the product information and the fashioninformation, and the user information.

In particular, the storage 230 may include a software module forextracting an image corresponding to a fashion item from productinformation collected from a shopping mall server and fashioninformation collected from a social network, analyzing the propertiesthereof, determining a fashion combination over a preset reference byperforming machine learning on the extracted image and the analyzedproperties, and creating a coordinated combination matched to thedetermined fashion combination on the basis of at least one of theproduct information and the fashion information, and the userinformation.

FIG. 18 is a diagram about software modules stored in a storage unitaccording to an embodiment of the present disclosure.

Referring to FIG. 18 , programs such as an image extraction module 231,a property analysis module 232, a machine learning module 233, a fashioncombination determination module 234, a coordinated combination creationmodule 235, a recommendation coordination creation module 236, etc. maybe stored in the storage 230.

Meanwhile, the operation of the processor 220 described above may beperformed by programs stored in the storage 230. Hereafter, detailedoperations of the processor 220 using programs stored in the storage 230are described in detail.

In detail the image extraction module 231 can extract a regioncorresponding to a fashion item from an image and automatically removethe background in the extracted region.

The property analysis module 232 can analyze the characteristic of theextracted region, that is, the kind, color, material, style, etc. of thefashion item.

The machine learning module 233 can analyze the kind, color, material,style, etc. of the fashion item included in fashion informationcollected from a social network and product information collected froman external server on the basis of the collected fashion and productinformation.

The fashion combination determination module 234 can determine a fashioncombination over a preset reference on the basis of preference and thenumber of times of sharing.

The coordinated combination creation module 235 can create coordinatedcombination in order of high degree of matching with the determinedfashion combination.

The recommendation coordination creation module 236 can create andsequentially provide the recommendable coordination information fromrecommendable coordination information satisfying user coordinationconditions.

As described above, software for performing the operation of the userterminal 100 may also be stored in the store in a module type, and canenable a processor included in the user terminal 100 to read out aprogram or software from the storage and enable the user terminal 100 toextract a region corresponding to a fashion item from an input image andcreate and provide user information including at least one of theproperties of the extracted region, a user shopping type, and usercoordination conditions.

Meanwhile, a non-transitory computer readable medium in which a programthat sequentially performs the control method according to the presentdisclosure is stored may be provided.

The non-transitory computer readable media is not media that stores datafor a short time such as a cache, and a memory, but media that cansemipermanently store data and can be read out by a device. In detail,the various applications or programs described above may be stored andprovided in a non-transitory computer readable medium such as a CD, aDVD, a hard disk, a blueray disc, a USB, a memory card, and a ROM.

A bus is not shown in the block diagram described in relation to theprocessor for extracting an image corresponding to a fashion item fromproduct information collected from a shopping mall server and fashioninformation collected from a social network, analyzing the propertiesthereof, determining a fashion combination over a preset reference byperforming machine learning on the extracted image and the analyzedproperties, and creating a coordinated combination matched to thedetermined fashion combination on the basis of at least one of theproduct information and the fashion information, and the userinformation, but communication between the components in the server maybe performed through a bus. Each device may further include a processorsuch as a CPU, a microprocessor, etc. that performs the various stepsdescribed above.

Further, although exemplary embodiments of the present disclosure wereillustrated and described above, the present disclosure is not limitedto the specific exemplary embodiments and may be modified in variousways by those skilled in the art without departing from the scope of thepresent disclosure described in claims, and the modified examples shouldnot be construed independently from the spirit of the scope of thepresent disclosure.

1. A system for constructing a virtual closet, comprising: a user terminal detecting an image including a user's face, extracting a region corresponding to at least one fashion item in the detected image, restoring the fashion item from the region corresponding to the fashion item, collecting a fashion image matched to the restored fashion item, and extracting and transmitting a characteristic of the collected fashion image and a creation date of the detected image to a server; and the server analyzing a wearing date and wearing frequency of the fashion item on the basis of the extracted date, detecting all images of the user wearing the fashion item, and storing the extracted characteristic of the fashion image, the wearing date of the fashion item, and the images of the user wearing the fashion item together with the fashion image, wherein the server creates a coordinated set image by grouping at least one fashion item included in the detected image, and stores a wearing date and frequency of the grouped fashion item, an image of the user wearing the grouped fashion item, and the created coordinated set together.
 2. (canceled)
 3. The system of claim 2, wherein the server provides information stored with the fashion item and information stored with the coordinated set image to the user terminal.
 4. The system of claim 3, wherein when the region corresponding to the fashion item and the creation date of the detected image are received from the user terminal, the server analyzes and updates in real time the region and the creation date.
 5. The system of claim 4, wherein the user terminal extracts purchase information and an image of a purchase product from a purchase history, and transmits the purchase information and the image.
 6. The system of claim 5, wherein the server extracts a characteristic of the image of the purchase product and stores the extracted characteristic and purchase information of the purchase product together.
 7. The system of claim 6, wherein the server analyzes similarity between the fashion item and the purchase product, replaces the fashion item with the purchase product and stores the information stored with the fashion item, the purchase information, and the purchase product together when the similarity is a preset reference or higher.
 8. The system of claim 7, wherein the server collects fashion information on the basis of preference and the number of times of sharing on a social network, creates a coordinated combination composed of at least one of fashion items and purchase products stored through the system for constructing a virtual closet on the basis of the collected fashion information, and provides the coordinated combination to the user terminal.
 9. A method of constructing a virtual closet, comprising: a step in which a user terminal detects an image including a user's face and extracts a region corresponding to at least one fashion item in the detected image; a step in which the user terminal restores the fashion item from the region corresponding to the fashion item and collects a fashion image matched to the restored fashion item; a step in which the user terminal extracts and transmits a characteristic of the collected fashion image and a creation date of the detected image to a server; a step in which the server analyzes a wearing date and wearing frequency of the fashion item on the basis of the extracted date; a step in which the server detects all images of the user wearing the fashion item; a step in which the server stores the extracted characteristic of the fashion image, the wearing date of the fashion item, and the images of the user wearing the fashion item together with the fashion image; and a step of creating a coordinated set image by grouping at least one fashion item included in the detected image, and storing a wearing date and frequency of the grouped fashion item, an image of the user wearing the grouped fashion item, and the created coordinated set together. 